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Abstract. For the group 0(p, q) we give a new construction of its minimal unitary 
representation via Euclidean Fourier analysis. This is an extension of the q = 2 
case, where the representation is the mass zero, spin zero representation realized in a 
Hilbert space of solutions to the wave equation. The group 0{p, q) acts as the Mobius 
group of conformal transformations on RP^^'^^i, and preserves a space of solutions 
of the ultrahyperbolic Laplace equation on R^'^-'^'^^i. We construct in an intrinsic 
and natural way a Hilbert space of solutions so that 0{p, q) becomes a continuous 
irreducible unitary representation in this Hilbert space. We also prove that this 
representation is unitarily equivalent to the representation on L-^(C), where C is the 
conical subvariety of the nilradical of a maximal parabolic subalgebra obtained by 
intersecting with the minimal nilpotent orbit in the Lie algebra of 0(p, q). 
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§1. Introduction 

1.1. In this paper we study the symmetries of the ultrahyperbolic Laplace operator 
on a real finite-dimensional vector space equipped with a non-degenerate symmetric 
bilinear form. We shall work in coordinates so that the operator becomes 

Uhp-1,9-1 = U2 :— - — 77 + ■ h 



on M"" = In the case of Minkowski space {q = 2) we are studying the wave 

equation, which is well-known to have a conformally invariant space of solutions, 
see [14]. This corresponds to the fact that the equation □jjp-i.q-i/ = in this 
case describes a particle of zero mass. Incidentally, it may also be interpreted as 
the bound states of the Hydrogen atom, namely each energy level corresponds to 
a i^-type - for (p, q) = (4, 2). This gives the angular momentum values by further 
restriction to 0(3). In general the indefinite orthogonal group G = 0{p, q) acts as 
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the Mobius group of meromorphic conformal transformations on ^'^ ^, leaving 
a space of solutions to nj^p-i,q-if = invariant. 

1.2. The main object of the present paper is to construct in an intrinsic and 
natural way a Hilbert space of solutions of nmp-Lq-i so that the action of 0{p, q) 
becomes a continuous unitary irreducible representation in this Hilbert space for 
{p, q) such that p,q > 2 and p + q > 4 is even. From an algebraic view point of 
representation theory, our representations are: 

i) minimal representations if p + q >8 (i.e. the annihilator is the Joseph ideal). 

ii) not spherical if p ^ q (i.e. no non-zero X-fixed vector). 

iii) not highest weight modules of SOq(j), q) if p,q > 3. 

In a long history of representation theory of semisimple Lie groups, it is only 
quite recent that our representations for p,q>3 have been paid attention, espe- 
cially as minimal unitary representations; they were first discovered by Kostant [13] 
for {p, q) = (4, 4) and generalized by Binegar-Zierau [3] as subrepresentations of de- 
generate principal series representations. There is also another algebraic approach 
to the same representations by using the theta correspondence for the trivial repre- 
sentation of S'L(2,R) by Huang-Zhu. Our previous papers [11] and [12] treated the 
same representation by geometric methods and with other points of view. We think 
that such various approaches reflect a rich structure of the minimal representations. 

It is perhaps of independent interest that the (in some sense maximal group 
of) symmetries 0(p, g) of the space of solutions of n^p-i,q-if = lead to such a 
natural Hilbert space. Our inner product ( , )w defined by an integration over 
a non-characteristic hyperplane (see (1.5.1)) is a generalization of the one coming 
from energy considerations in the case of wave equations, and even the translation 
invariance of the inner product contains some new information about solutions. 

It is also of independent interest from the representation theory of semisimple 
Lie groups that our representations are unitarily equivalent to the representations 
on L^(C), where C is the null cone of the quadratic form on Mp~^''^~^. This 
result is proved via the Fourier transform in Theorem 4.9. Such L^-realizations 
of "unipotent representations" is expected from the philosophy of the Kostant- 
Kirillov orbit method, but has not been proved except for some special cases of 
highest weight modules or spherical representations. 

We have avoided most of the references to the theory of semisimple Lie groups 
and representation theory, and instead given direct constructions of the key objects, 
such as for example the minimal K-type; this is given as an explicit hypergeometric 
function, and we also calculate its Fourier transform in terms of a Bessel function. 
By application of explicit differential operators forming the Lie algebra of G we can 
generate the whole Hilbert space of solutions beginning from the minimal K-type. 

1.3. For g = 2 (or 7? = 2) we are dealing with highest weight representations (when 

restricting to the identity component S0o{p,2)), and these have been studied by 
many authors, in particular in the physics literature. For a nice introduction to this 
representation and its construction via geometric quantization (and more) see [8]. 
In this case the K-types may be identified with energy levels of the bound states 
of the Hydrogen atom, and the smallest one with the bound state of lowest energy. 

We can summarize the situation, covering both the classical Kepler problem and 
its quantization in case g = 2, as in the diagram below. Here the left-hand side 
represents the classical descriptions of respectively the Kepler problem and geodesic 
flow on the sphere; by "symplectic transform" we are alluding to the change of 
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variables between these two Hamiltonian systems as presented in [8]. The right- 
hand side involves the quantizations of these two systems, where the wave-equation 
is considered as the quantization of geodesic flow, also to be thought of as geometric 
optics. The quantum analogue of the "symplectic transform" involves the Fourier 
transform. Finally we invoke conformal geometry and combine it with the Fourier 
transform, which in a different (and new as far as constructing Hilbert spaces and 
unitary actions) way appears in passing from the wave equation to the Fourier 
realization of solutions - this is the last arrow on the right-hand side. 



Classical Quantum 



Kci)lcr i)rol)lcm 



Hj'drogcii atom 



J| symplectic transform 



geometric optics 



J| Fourier transform 



(A) wave equation 



conformal geometry 
-IJ. Fourier transform 



(B) other realizations and 
explicit inner products 
of minimal representations 



The main focus of this paper is on the boxes (A) and (B). In particular, we give an 
explicit inner product in the model (A) (Theorems 1.4 and 1.5) and construct via 
Fourier transform a new realization of the minimal representation (Theorem 1.6) 
for general p, q. 

1.4. From now, suppose that n :— p + q — 2 is an even integer greater than 2, and 
p,q > 2. Let us briefly state some of our main results in a more explicit way. 

First, we find a formula of the Green function £"0 of the ultrahyperbolic Laplace 
operator D^p-i.^-i, in Proposition 4.2, namely, Eq is given by a constant multiple 
of the imaginary part of the regularized Schwartz distribution: 

e 2 (^xi^ + ■ ■ ■ + Xp-l^ - Xp^ Xp+q-l^ + V^Oy 2. 

See the recent paper of Hormander [9] for further details on distributions associated 
with this ultrahyperbolic equation. Then we construct solutions of H^^p-i.q-if = 
by the integral transformation: 

S : C^{W) C^iW"), ^^Eo*<f (see (4.3.1)). 

The image S{C^{W)) turns out to be "large" in Ker Drp-i.,-! (see §4.7, Re- 
mark (2)). On this image, we define a Hermitian form ( , )jv by 



(1.4.1) (/i,/2)jv:=/ / Eo{y - x)(pi{x)(p2{y) dxdy, 

where fi — Eq* ipi {i — 1, 2). Here is a part of Theorem 4.7, which is the first of 
our main results: 
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Theorem 1.4. ( , )n is positive-definite on the image of S . Furthermore, 0(p,q) 
acts as an irreducible unitary representation on its Hilbert completion H. 

We shall write (tx7™p"i 7i) for this unitary representation. We also prove 
that this representation is isomorphic to the minimal representation of 0{p,q), 
constructed previously by Kostant, Binegar-Zierau ([13], [3]). Thus, Theorem 1.4 
may be regarded as a realization of the minimal representation (with an explicit 
inner product) in the solution space of the ultrahyperbolic equation. 

1.5. The above definition of the inner product ( , ) at (see (1.4.1)) uses the integral 
expression of solutions of Drp-i,?-!/ = 0. Can we write the inner product without 

knowing the preimage? Yes, the second of our main results is to give an intrinsic 
inner product on the same solution space by using the Cauchy data. For simplicity, 
we take 2;i = as a non-characteristic hyperplane. Then, we decompose a solution 

/ = /+ + /- 

such that f±izi, . . . , Zn) is holomorphic with respect to the first variable zi in the 
complex domain of {zi G C : zblm^i > 0} of the ^i-variable. This is an expression 
of / as a hypcrfunction, and such a pair (/^., /_) can be obtained by the convolution 
in the ^i-variable (see (6.2.3)): 

f±{z) = ^ ■ ^"L^ * f{zi, . . . , z^), 
27rv— 1 zi ^ V— 10 

where the integration makes sense for / with suitable decay at infinity. Then we 
define a Hermitian form 



(1.5.1) (/,/). := (/,|^± -/_|.) 

Then we shall prove that ( , )vf is independent of the specific choice of a non- 
characteristic hyperplane, as follows from the (non-trivial) isometric invariance. 
Much more strongly, ( , )vk is conformally invariant. A precise formulation for this 
is given in Theorem 6.2, which includes: 

Theorem 1.5. 47r( , )w = { , )n- In particular, ( , )w is positive definite and 
0{p, q) -invariant. 

Hence, in place of Theorem 1.4, we can obtain the same irreducible unitary 
representation of 0{p, q) on the Hilbert completion of a space of solutions with 
respect to the inner product ( , )w- 

An interesting property of this inner product is its large invariance group. Even 
in the case of the usual wave equation {q = 2 case) our approach to the Hilbert 
space of solutions and the corresponding representation offers some new points of 
view. In this case, if we take the non-characteristic hyperplane as fixed time coordi- 
nate (namely, if zi stands for the time), then the translational invariance amounts 
to a remarkable "conservation law". Instead, we can take the non-characteristic 
hyperplane by fixing one of the space coordinates, and an analogous integration 
over the hypersurface (containing the time coordinate) still gives the same inner 
product ! As a final remark in §6.7, we note the connection to the theory of con- 
served quantities for the wave equation {q — 2 case), such as the energy and others 
obtained by the action of the conformal group. 
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1.6. The Gelfand-Kirillov dimension of our representation (tJ7™p"^i 7i) is p + 
g — 3. So, we may expect that the representation could be realized on a, {p + q — 3)- 
dimensional manifold. For this purpose, we define the null cone of the metric as 

C := {C e R" : Ci' + • • • + Cp-i' - Cn' = 0}. 

The third of our main results is another realization of the unitary representation 
{'U!j^p^i^q_i, Ti) in a function space on a (p + g — 3)-dimensional manifold C. The 
Fourier transform maps solutions of Drp-i.^-i/ = to distributions supported 
on the null cone C. Surprisingly, the inner product of our Hilbert space turns out 
to be simply the L^-norm on C with respect to a canonical measure d/j, (see (3.3.3)) 
! Here is a part of Theorem 4.9: We regard L'^{C) as a subspace of distributions 
by a natural injective map T : L'^iC) — > 4S'(M"^). 

Theorem 1.6. (27r)-tT-^ oT is a surjective unitary operator from H to L'^{C). 

Theorem 1.6 defines an irreducible unitary representation of G = 0{p,q) on 
L?{C), denoted by tt, which is vmitarily equivalent to (tx7™p"^i,q_i, 7i). Since the 
maximal parabolic subgroup P™^^^ of G (see §2.7) acts on R^^^''^^^ as affine trans- 
formations, the restriction 7r| pmax has a very simple form, namely, the one obtained 
by the classical Mackey theory (see (3.3.5)). In this sense, Theorem 1.6 may be 
also regarded as an extension theorem of an irreducible unitary representation 
from the maximal parabolic subgroup P™*^ to the whole group G. 

1.7. The fourth of our main results is about the representation (tt, L^(C)) as 
(g, -fir)-modules on the Fourier transform side, especially to find an explicit vector 
in the minimal i^T-type. 

In the realization on L^(C), the action iT{g) is not simple to describe except for 
g g pmax Instead, we consider the differential action dir of the Lie algebra go 
on smooth vectors of L'^{C), which turns out to be given by differential operators 
at most of second order (see §3.2). This makes the analogy with the metaplectic 
representation (where G is replaced by the symplectic group) a good one . Here we 
are recalling the fact, that the even part of the metaplectic representation may be 
realized as an L^-space of functions on the cone generated by rank one projections 
in R". 

Moreover, by using a reduction formula of an Appell hypergeometric function, 
we find explicitly the Fourier transform of a Jacobi function multiplied by some 
conformal factor which equals to a scalar multiple of 

iPoAO ■■= \C\'^K^{2\C\)dfi e 5'(R-). 

Here K,y{Q is a modified Bessel function of the second kind. This vector 'i/'o,e(C) 
corresponds to the bound state of lowest energy for q = 2 case . For general p, q, 
the iiT-span of ipo,e{0 generates the minimal K-type in the realization on L'^[C). 
We define a subspace U of 5'(R"^) to be the linear span of its iterative differentials 

dTT{Xi) ■ • •(i7r(Xfc)V'0,e(C) (^1, • • • e 00 ®M C). 

What comes out of §5 may be formulated in this way (combining with Theorem 
4.9, see §3.2 for notation): Suppose p + q E 2Z, p + q > A and p> q>2. 
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Theorem 1.7. 1) \C\~^Kq-z (2|C|) is a K-finite vector in L'^{C). 

2) U is an infinitesimally unitary {q, K)-module vxa uj 71—2 , 

3) IJ is dense in the Hilbert space T{LF'{C)). 

4) The completion of (2) defines an irreducible unitary representation of G on 
T{L'^{C)), and then also on L^{C). 

In the paper [4] one finds a similar construction of Hilbert spaces and unitary 
representations for Koecher-Tits groups associated with semisimple Jordan algebras 
under the assumption that the representations are spherical, and there also occur 
Bessel functions as spherical vectors. In our situation the representations are not 
spherical if p ^ q. Our approach is completely different from [4], and contrary to 
what is stated in [4] (p. 206) we show that for G — 0(j), q) it is possible to extend 
the Mackey representation of the maximal parabolic subgroup to the whole group. 
Furthermore, even foTp = q case, our approach for Theorem 1.7 has an advantage 
that we give the exact constants normalizing the unitary correspondence between 
the minimal i^-type in other realizations and the Bessel function in our realization 
on L?{C) (see Theorem 5.5). 

1.8. The paper is organized as follows: We begin by recalling some results from 
conformal geometry and facts about the conformal group, in particular in [11]. In 
section 3 we give the basic setup for a realization on the null cone via Fourier trans- 
form. Then we construct the intertwining operator from the minimal representation 
to the model treated here and calculate the new expression for the inner product 
(see Theorem 1.4). We show in Proposition 4.2 that the Green function of Okp-i,?-! 
has a Fourier transform equal to the invariant measure on the null-cone, allowing 
one more expression for the inner product (see Theorem 1.6); also we obtain from 
this an intertwining operator from test functions to solutions. Indeed, in section 
4 Proposition 4.6 we prove that the Green function is up to a constant exactly 
the kernel in the Knapp-Stein intertwining integral operator between degenerate 
principal series representations at the parameters we study; this makes the proof of 
unitarity of the minimal representation elementary and formulated independent of 
the semi-simple theory usually employed here. Note that all normalizing constants 
are computed explicitly. Lemma 2.6 states the irreducibility and unitarizability, 
which we use; we give in [12] independent proofs of these facts. 

In section 5 we construct the lowest i^-type as a modified Bessel function, whose 
concrete properties are important for K-type information about L^(C). The idea 
here is to use a classical formula on the Hankel transform due to Baily in 1930s, 
and then apply reduction formulae of an Appell hypergeometric function of two 
variables. 

Section 6 contains formulae for the inner product ( , )vk in terms of integration 
over a Cauchy hypersurface. Summarizing, we give five different realizations of 
the inner product together with the normalizations of these relative to each other. 
Namely, in addition to ( , )n and ( , )vi/ we also define three more: ( , )m (coming 
from a pseudo differential operator on M = S'p~^ x S'^~^), { ^ )a (coming from a 
normalized Knapp-Stein intertwining operator), and finally ( , )c, which is just 
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(C). This is seen in the key diagram (see Section 4.11) 



(C) 



n 



KerD 



where the spaces correspond to four different ways of generating solutions to our 
ultrahyperbohc equation. S will be an integral transform against the Green kernel 
(essentially, a Knapp-Stein intertwining operator with a specific parameter), and 
T the Fourier transform, mapping solutions to distributions supported on the null 
cone C. Correspondingly to the various ways of generating solutions, we write 
down explicitly the unitary inner product and its Hilbert space. We have tried 
to avoid the use of any semi-simple theory and stay within classical analysis on 
spheres and Euclidean spaces; still our treatment may also be of interest to people 
working with the classification of the unitary dual of semi-simple Lie groups, since 
we are providing new models of some unipotent representations. Tools like the 
standard Knapp-Stein intertwining operators become very natural to use here, also 
from the more elementary viewpoint, and the close connection between these and 
Green functions for ultrahyperbohc differential operators seems not to have been 
noticed before. 

The first author expresses his sincere gratitude to SDU - Odense University for 
the warm hospitality. 

§2. Ultrahyperbolic equation on Rp~'^'1~'^ and conformal group 

2.1. As explained in the Introduction, we shall give a fiat picture, the so-called N- 
picture, of the minimal representation, which is connected to classical facts about 
conformal geometry in R". We shall give a unitary inner product in this realization 
(see Theorem 6.2) and also in its Fourier transform (Theorem 4.9), together with 
an explicit form of minimal i^T-type in this realization (see Theorem 5.5). 

We shall assume p+q e 2N, p > 2, q > 2 and (p, q) 7^ (2, 2). The parity condition 
p + q E 2N is not necessary when we consider a representation of the parabolic 
subgroup pmax or of the Lie algebra Q. Indeed, it will be interesting to relax this 
parity condition in order to obtain an infinitesimally unitary representation, which 
does not integrate to a global unitary representation of G. 

Throughout this paper, we let 



This section is written in an elementary way, intended also for non-specialists of 
semisimple Lie groups. §2.2 and §2.6 review the needed results in [11]. 

2.2. We recall some basic fact of the distinguished representation of a conformal 
group (see [11], § 2). Let M be an n-dimensional manifold with pseudo-Riemannian 
metric qm- We denote by Am the Laplace operator on M, and by Km the scalar 
curvature of M. The Yamabe operator is defined to be 



n = p + q — 2 



Am '■— Am — 



n-2 



Km. 



4(n - 1) 
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Suppose (MjQm) and {N^g^) are pseudo-Riemannian manifolds. A local diffco- 
morphism $: M — is called a conformal map if there exists a positive- valued 
function Q on M such that ^*gN — ^^S'm- For A e C, we introduce a twisted 
pull-back 

(2.2.1) ^l: C°°{N) ^C°°{M), f ■ f o^. 

Then the conformal quasi-invariance of the Yamabe operator is expressed by: 

(2.2.2) $*^Ajv = AM^t^. 

2 2 

Let G be a Lie group acting conformally on M. If we write the action as x ^ L^x 
{h E G,x e M), we have a positive function VL{h, x) E C^{G x M) such that 

LIqm = ^{h, -f Qm {h e G). 
We form a representation wx of G, with parameter A e C, on C"^(M) as follows: 

(2.2.3) wxih-^)fix) = ^{h,xff{Lhx), (heGJe C°^{M),x e M). 

Note that the right-hand side is given by the twisted pull-back {Lh)\ according to 
the notation (2.2.1). Then, Formula (2.2.2) implies that Am: C°°(M) C°°{M) 
is a G-intertwining operator from zun^ to zon+2. Thus, we have constructed a 

2 2 

distinguished representation of the conformal group: 

Lemma 2.2 (see [11], Theorem 2.5). Ker Am is a representation space of the 
conformal group G of a pseudo-Riemannian manifold {M,gM), through Wn-2 . 

If {N,gN) is also a pseudo-Riemannian manifold on which the same group G 
acts conformally, then one can also define a representation w\^n on C°°{N). Then 
the twisted pull-back is a G-intertwining operator. 

2.3. Here is a setup on which we construct the minimal representation of 0(p, q) 
by applying Lemma 2.2. Let p,q>2. We note n = p-\-q—2. We write {eo, . . . , Cn+i} 
for a standard basis of W'^'^ and the corresponding coordinate as 

{vo, . . . , Vn+l) = (x, y) = (vo, z', z", Vn+l), 

where x G M.P,y G W,z' G W~^,z" G The notation {x,y) will be used for 

SP-i X Si-^, while {z' , z") for = m(p-i)+('?-i). The standard norm on will 
be written as | • | (/ = p — l,p, g — 1, g). 

We denote by IR^'^ the pseudo-Riemannian manifold W'^'^ equipped with the flat 
pseudo-Riemannian metric: 

(2.3.1) ^fRP.q = dvo' -I h dvp-x' - dv^ dvn+x' ■ 

We put two functions on W'^'^ by 

(2.3.2) zy:M^'+''^M, {x,y)^\x\, 

(2.3.3) /x:R^'+«^]R, (^;o, . . . , ^;^+i) ^ ^(^;o + ^;„+i). 
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and define three submanifolds of R^'^ by 



M 
N 



= {(x,y)GRP''^:|x| = |2/|^0}, 
= {ve RP'"^ : ii{v) = 1} n H 



b 



where the bijection l : N is given by 



(2.3.4) r.R^^N, {z' , z") ^ (1 - ' ' ^' ' , z' , z\ 1 + ' ' ^ ' )■ 
We say a hypersurface L of S is transversal to rays if the projection 



(2.3.5) ^:E^M,v^ 



V 



iy{v) 



induces a local diffeomorphism L — > M. Then, one can define a pseudo- 

Riemannian metric gi of signature (p — 1,^ — 1) on L by the restriction of g«,p,q. 
In particular, M itself is transversal to rays, and the induced metric ggp-iy^gq-i 
equals gsp-^ © ("^s^-O; where gs^^-i denotes the standard Riemannian metric on 
the unit sphere 5"""^. Likewise, the induced pscudo-Riemannian metric on 
through l: ^ MP''^ coincides with the standard fiat pseudo-Riemannian metric 
on W. 



2.4. Let Ip^q := diag(l, . . . , 1, -1, . . . , -1) e GL{p + q,R). The indefinite orthog- 
onal group 

G = Oip, q) := {g E GL{p + q,R) : 'glp,,g = J^, J, 

acts isometrically on MP''^ by the natural representation, denoted hy v g ■ v. 
This action stabilizes the light cone S. We note that the multiplicative group 
R+ := {r e ]R : r > 0} also acts on S as dilation, which commutes with the linear 
action of G. Then, using dilation, one can define an action of G on M, and also a 
meromorphic action on M^~^'^~^ as follows: 

(2.4.1) Lh-.M^M, y^J}-^ (heG), 

(2.4.2) Lh : W-^''i-^ ^ E^-^'^-\ z ^ r"- ( {h G G). 



Ii(h-L{z)) 



Then, both of these actions are conformal: 



(2.4.3) {LhrgM = i^{h-v)-^gM 

(2.4.4) {Lh)*gRP-i.<!-i = /x(/i • L{z))~'^gRp-i,q-i. 

We note that (2.4.2) and (2.4.4) are well-defined if ii{h ■ l{z)) ^ 0. In fact, G 
acts only meromorphically on RP"^''^'"^. An illustrative example for this feature 
is the linear fractional transformation of 5'L(2,C) on P^C = C U {oo}, which is 
a meromorphic action on C This example essentially coincides with (2.4.2) for 
{P-i q) — (3, 1), since SL{2, C) is locally isomorphic to 0(3, 1) and C ~ R^. 
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2.5. The (meromorphic) conformal groups for the submanifolds M and NofS are 
the same, namely, G = 0{p, q), while their isometry groups are different subgroups 
of G, as we shall see in Observation 2.5. In order to describe them, we define 
subgroups M"^^^, iV"^^^, A"^^^ and of G as follows: 

First, we set 



mo 
K 

^max 
j^^max 



p+qj 



= GnOip + q) = 0(p)xOiq), 

= {g e G : g ■ eo = eo, g- e„+i = Cn+i} ^ 0{p -l,q-l), 

^^max^^^^max ~0(p - 1, ? - 1) X Z2. 



The Lie algebra of G is denoted by Qq — o{p, q), which is given by matrices: 

Qo-{Xe M{p + q,R): Xlp^g + = O}. 

Next, we keep n = p + g — 2in mind and put 

r 1 (1< 7 < » - 1), 
(2.5.1) Ej^l ) ~ \ 

and define elements of Qq as follows: 



(2.5.2) (a) Nj := Ej^o + ^j,n+i - SjEoj + ejE^+ij (1 < j < n), 
(2.5.2)(b) N, := Ej,o - i?,,n+i - £,i?o,, - £,i?n+i,, {I < J < n), 

(2.5.2) (c) E := Eo,n+i + E^+i^o, 

where Eij denotes the matrix unit. Now, we define abelian subgroups of G by 

n n 

:= exp(^ RNj), iV™^^ := exp(^ RNj), yl'"^^ := exp(RE). 
i=i j=i 

For example, M^^^ is the Lorentz group and M^^^N^^^ is the Poincare group if 
{Pi q) = (2, 4). It is convenient to identify with N'^^^ by putting 

n 

(2.5.3) na := exp(J^ajiV^) e ]V°^ for a = (ai, . . . , a^) e M". 

The geometric point here will be the following: 

Observation 2.5. 1) On Sp~^ x S'^~^, G acts conformally, while K isometrically. 
2) On MP~^''i~^, G acts meromorphically and conformally, while the motion group 
j^^max^max isomctrically. 

2.6. Next, let us consider the pseudo-Riemannian manifold M = S^~^ x It 
follows from (2.3.3) and (2.4.3) that we can define a representation wx^m of G on 
C°°(M) by 

{wxMh-')f){v) ■■= Hh ■ v)-^f{Lhv), 
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The Yamabe operator on M is of the form: 

Am = A5P-1 - Agq-i - ^ )^ + ^ )^ = A5P-1 - AsQ-i. 

Applying Lemma 2.2, we obtain a representation of the conformal group G = 
0(p,q), denoted by (tu^'^, F^'^), as a subrepresentation of W p+g-A. 

2 ' 

VP''^ := Ker Am = {/ e C~(M) : Am/ = 0}, 

{wP''i{h-^)f){v) ■.^u{h-v)-'^f{Lhv), for he G,veM,f e VP''^. 

The restriction of ti7^'^ from the conformal group to the isomctry group gives a 
useful knowledge on the representation ro^'"^. For this, we recall the classical theory 
of spherical harmonics, which is a generalization of Fourier series for S^. For p >2 
and A; e N, we define the space of spherical harmonics of degree k by 

(2.6.1) n\RP) = {fe C^iSP-') : Asp-if=-k{k + p-2)f}, 

= {/ G C-{SP-') : Asp-.f + /}. 

Then 0{p) acts irreducibly on H'^iW') and the algebraic direct sum ©^o^'^(^^) 
is dense in C°°{SP-'^). We note that n''{M.'^) + {0} only if /c = or 1. 

Now, we review a basic property of this representation (^17^'^,^^'^) on M = 
^P-i X 5^-1: 

Lemma 2.6 (sec [3]: [11], §3). Assume p, q > 2, p + q e 2N and (p, q) 7^ (2, 2). 

1) (ro^''^, yP''^) an infinite dimensional irreducible representation ofG. 

2) (K-type formula) V^'^ contains the algebraic direct sum 

(2.6.2) H''{W)®n\W) 

o+f=6+f 

as a dense subspace with respect to the Frechet topology on C°°{M). 

3) G preserves the norm on V'^''^ defined by 

\\F\\m ■■= Wil - A5.-0^^lli^(M) = E + ^^)\\Fa,b\\hiMy 

a>max(0,^^y^) 

if F = J2aFa,b e VP^i with Fa,b G 7i«(Rf) 7Y''(M«) and b = a + Here, 
(| — Agp-i)3 zs a pseudo-differential operator on M , which is equal to {\ — Agi-i)^ 
on Ker Am- 

We write ( , )m for the corresponding inner product. We denote by F^''^ the 
Hilbert completion of V^^'^, on which G acts as an irreducible unitary representation 
of G. We shall use the same notation w'^''^ to denote this unitary representation. 

lip> q then "l/^''? contains the i^-type of the form IKITi^^ (M^). This i^-type is 
called a minimal X-type in the sense of Vogan, namely, its highest weight (with 
respect to a fixed positive root system of to) attains the minimum distance from 

11 



the sum of negative roots of to among all highest weights of K-types occurring in 
tu^''^. Likewise for p < q. 

Remark. 1) If p + q > 8, zu^''^ is called the minimal representation in the repre- 
sentation theory of semisimple Lie groups, in the sense that the annihilator is the 
Joseph ideal. 

2) The formula (2.6.2) is regarded as a branching law from the conformal group G 
to the isometry subgroup K of the pseudo-Riemannian manifold M — S^~^ x S'^~^ 
(see Observation 2.5). In [12], we generalized this branching law with respect to 
a non-compact reductive subgroup and proved the Parseval-Plancherel formula, in 
the framework of discretely decomposable restrictions [10]. 

2.7. Let us consider the flat pseudo-Riemannian manifold Rf~^''3'~^. The Yamabe 
operator on R^'"-'^'^"^ is of the form: 

□rp-1,9-1 = := - — 9 + ■ ■ ■ + 



because the scalar curvature on IR.^"^''^'"^ vanishes. Since G = 0{p,q) acts on 
ag a (meromorphic) conformal transform by (2.4.4), we obtain a 'repre- 
sentation' with parameter A e C as in (2.2.3): 

(2.7.1) wx,e,R49~')fiz) = H9i{z))\-\,{sgn{i,{gi{z))))f{Lgz), {g E G). 
Here, for e = ±1, we put 

Xe:M^ ^{±1} 

by xi = 1 and X-i = sgn. We may write WA,e,KJ'-i'«-i for t<7A,e,K" if we emphasize 
a view point of conformal geometry on the flat space R^*"^'^"^. 

We note that C°°(M) is not stable by 'cox,e,m.p-^'i-^{9~^) because Lg is mero- 
morphic. To make (2.7.1) a representation, we need to consider suitable class of 
functions controlled at infinity. One method for this is to use a conformal compact- 
ification 

and to take a twisted pull-back from C°°{M) by a conformal map This 
method is easy, and we shall explain it soon in §2.8 and §2.9. The other is to 
find an inner product for specific parameter A so that G acts as a continuous 
unitary representation on the Hilbert space. This is particularly non-trivial for a 
subrepresentation, and we shall consider it for Ker D^p-i,,-! in §6. 

Before taking a suitable class of functions, we first write a more explicit form of 
(2.7.1). First, we note that the maximal parabolic subgroup 



pmax ^max^max^max = (^x x 0{p-l,q-l) X Zg) K 

acts transitively on the manifold t(R") as afiine transformations. Furthermore, 
j^^maxjymax ^cts On i(R"') as isometrics (see Observation 2.5). Correspondingly, the 
representation wx^^ = 'UJx^^^]^n. given in (2.7.1) has a simple form when restricted to 
the subgroup P^ax. 

(2.7.2)(a) {wx,e{m)f){z) = f{m-'z) (m e M™^-), 

{wx,e{mo)f){z) = ef{z), 

(2.7.2)(b) (wA,e(e*^)/)(z) = e^'fie'z) (t e R), 

(2.7.2)(c) {wx,e{na)f){z) = f{z - 2a) (a G R^). 
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Second, we write an explicit formula of the differential action of (2.7.1). We 
define a linear map 

by the Lie derivative of the conformal factor z) := fj,{h ■ l{z))~^ (see (2.4.4)). 
For Y = {yi,j)o<i,j<n+i e 00 and z eW, we have 

d 1 ^ 

(2.7.3) uiY), := -\,^,n{e'^ , z) = -r^n+i - -J^i^o,, + Yn+,j)zj. 

We write the Euler vector field on MJ^ as 

(2.7.4) ^-t-.^. 

Then the differential dtux : go ^ End(C°°(M'^)) is given by 

(2.7.5) dwxiY) = -Xoj{Y) - a;(y)E^ 

""I I '|2 I "|2 \ f) 

- E { (^^0 + + ' ~ ' (-y»,o + i^i,n+i) + 5^ 

for y = (li,j)o<i,7<n+i £ 00 and z e R"'. In particular, we have 



dzi 



(2.7.6) dwx{Nj) = -Xejzj - SjZjE, + ^(|2;f - l-^'f )^, (1 < J < n). 



2.8. We recall M = Sp~^ x This subsection relates the representation 

tux^M and zux,mp-i,Q-i by the stereographic projection : M — > M^'"-'^'^"-'^ defined 
below. 

We set a positive valued function r : M" ^ E by 

r{z) = t{z' , z") :— f o i[z) 



lr'|2 _ |r"|2 \ h 

= ((!- ' ' J ' r + l^f 

1 

|.y'|2 _ |.y"|2 \ 2 

(!+ ' ' J ' + 

(2.8.1) = (^1 + (^^^)^) ' (l + (-'''' " ' 

We define an injective diffeomorphism as a composition of l: R^'"-'^'^"-'^ ^ S (see 
(2.3.4)) and S ^ M (see (2.3.5)): 

(2.8.2) * : ^ M, z ^ t{z)-^ i{z). 
The image of ^ is 

(2.8.3) M+ := {u = (uq, m', m", Wn+i) G M = S'^'"^ x : uq + > 0}. 
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Then, is a conformal map (see [11], Lemma 3.3, for example) such that 

(2.8.4) **5rM = T{z)-^g^r-i,,-i. 
The inverse of RP~^'«~^ M+ is given by 

(2.8.5) ^-\uo,u',u",Un+i) = C'^^''+' r\u',u") = i,{u)-\u',u") 

is nothing but a stereographic projection if g = 1. We note that ^ induces a 
conformal compactification of the flat space M^"^'^"^: 

Here ~ Z2 denotes the equivalence relation in the direct product space S'p~^ x S'^~^ 
defined by w ~ —u. 

As in (2.2.1), we define the twisted pull-back by 

(2.8.6) : C~(M) ^C~(M"), F ^ t{z)-^F{^{z)). 
Let 

C^{M), := {/ e C^{M) : f{-u) = ef{u), for any u G M}. 
Then '^*x\c^{m)^ is injective. The inverse map is given by 

(2.8.7) {^IX^ • *a(C°° We) ^ C^{M),, 

|Ho±Hn±i|-Ay(^-i(^)) («eM+) 

e r°+^"+^ |-^/(^-^(-n)) (wgM_). 

We note that (\&^ e)~^f makes sense for / G C(f^(M"^), since we have 

Co~(R") C ^t(C°°(M),). 

Now, the representation WA,e,]R» is well-defined on the following representation 
space: ^'^(C°°(M)), a subspace of C°°(M'^) through wx,m- 
Then, by (2.2.2) (see [11], Proposition 2.6), we have: 

Lemma 2.8. ^'*^(FP''^) C Ker Okp-i,,-!, where VP'I = Ker Am- 
2 

2.9. In the terminology of representation theory of semisimple Lie groups, '^1 is 
a G-intertwining operator from the K-picture {wx^m,C°°{M)^) to the N-picture 
(^A,e,M", \I/^(C°°(M))). To see this in an elementary way, we argue as follows: For 

G C, we denote by the space 

(2.9.1) 5^(S) := {h G C~(S) : h{tO = r/i(0, for any ^ G S,t > 0}, 

of smooth functions on S of homogeneous degree u. Then G acts on 5^(2) by left 
translations. Furthermore, for e = ±1, we put 

(2.9.2) S'-'^E) := {h G S^{E) : h{-0 = eh{0, for any ^ G S}. 
Then we have a direct sum decomposition 

on which G acts by left translations, respectively. Then 5"^'^(S) corresponds to the 
degenerate principal series representation (see [11] for notation): 

(2.9.3) C°°-Ind^„.ax(e (g) Ca) ~ S-^-^^'{E), 

where P™^^ = j^^max^maxjymax^ 

14 




Lemma 2.9. 1) The restriction S~'^''^{'E) C°°{M)^, h h\M induces the iso- 
morphism of G-modules between and (ota, C°°(M)e) for any A G C. 
2) The restriction S~'^''^{'E.) C°°(M"'),/i t-^ /ilmn induces the isomorphism of G- 
modules between S~^'^(E) and (roA,e,M", \I/^(C°°(M)e)) for any A e C. 

Proof. See [11], Lemma 3.7.1 for (1). (2) follows from the commutative diagram: 

(2.9.4) n / \ rs 

C~(M)e ^ C°°(M"), 
because ri is bijective and r2 is injective. □ 

2.10. A natural bilinear form ( , ) : S~^~^{'E) x S'^~^(S) — > C is defined by 

(2.10.1) (/ii,/i2) := / hi{b)h2{b) db 

JM 

(2.10.2) =2 I hi{i{z))h2{i{z)) dz (see (2.3.4)). 

Here, db is the Riemannian measure on M ^ S^-'^ X S'i-'^ . The second equation 
follows from {hih2){i{z)) = T{zy{hih2){'^{z)) and the Jacobian for M*^ ^ M 
is given by t{z)~'^ (see (2.8.4)). Then ( , ) is X-invariant and A^™^^-invariant from 
(2.10.1) and (2.10.2), and thus G-invariant since G is generated by K and N"^^^. 

§3. Square integrable functions on the cone 

3.1. In this section we shall study the irreducible unitary representation of the 
motion group M™^^A^™^^ ~0(p— l,g — l)x Mf+'^~^ and the maximal parabolic 
subgroup P™ax _ j^^max^maxjy-max ^]^g space of solutions to our ultrahypcrbolic 
equation n^p-i,q-if = 0. This is a standard induced representation by the Mackey 
machine, and will be later extended to the minimal representation of G = 0{p, q) 
(see Theorem 4.9 (3)). 

3.2. In the flat picture R^"-'^ '?"^, our minimal representation V^''^ of 0{p, q) can 
be realized in a subspace of Ker niKp-i,?-! (see Lemma 2.8). We shall study the 
representation space by means of the Fourier transform. 

We normalize the Fourier transform on 5(]R") by 

(^/)(C)= / f{z)e'^^'^^^+-+'-<-Uz,---dzn, 

and extends it to iS'(M"^), the space of the Schwartz distributions. 
By composing the following two injective maps 

we define a representation of G and Q on the image .F^* -2 (C°°(M)), denoted 

2 

by ^A,e = ^A,e,]R") SO that !F o ^* _2 is a bijective G-intertwining operator from 

' ' ' 2 
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the representation space {'U!Jx,M,C°°iM)e) to {Bx,e, ^^*n^{C°° {M)^)). Then, it 

' ' 2 

follows from (2.7.2) that the representation WA,e has a simple form when restricted 



to the subgroup P'^a^ = M"^*''yl™^''7V™ax. 

(3.2.1)(a) (^A,.(m)/i)(C) = hCmQ (m e M™^^), 

{wx,e{mo)h){C) = eh{C), 

(3.2.1)(b) {BxAe''')hm = e'^^-^^'h{e-'0 (t G M), 

(3.2.1)(c) (aA,e(na)/i)(C) = e2v^(«i^^+-+""^")/i(C) (a e R^). 

We remark that in the above formula, we regarded /i as a function. The action of 
^max space of distributions is slightly different by the contribution of the 

measure d(: 

(3.2.1)(b') {wxAe'^'mO = e"V(e-*C) {t e M), 

if we write (piC) = h(C)dC e 5'(M'^). 

The differential representation dwx^,. of Qq on J^^'V^ ((^""(M)) is given by the 

' 2 

following lemma: 

Lemma 3.2. PFe reca// that E^^ is the Euler operator (see (2.7.4-)) ■ With notation 
in (2.5.2), we have 

dwx,e(Nj) = 2v^0 (1 < J < n), 

dejxANj) = (a£,^ - Ece~^ + ^CPc) (1 < i < 
dwx,e{-^) = A — n — i?^. 



Proof. Lemma follows from the correspondence under the Fourier transform -t^- 



9 ^ 



•3 



— V— lCj5 ^3 ^ — V— and therefore from i?;^ ^ — n — -P(-2) -De where 
-P(^) := ^1^ + • • ■ + Zp-i^ — Zp^ — ... — □ 

Remark. 1) We note that dwx,e is independent of the signature e = ±1. 
2) In Theorem 4.9, we shall find that L^(C), the Hilbert space of square integrable 
functions on the cone C in R"^, is a G-invariant subspace of the Schwartz distri- 
butions iS'(M"'). Then, the action of the Lie algebra g can be written in terms of 
differential operators along the cone C at most of second order. 

3.3. We define a quadratic form Q on W (~ (R^)*) as the dual of P{z) on R" by 

(3.3.1) Q(C) Ci' + • • • + Cp-i' - for C e R^ = W^'^-^ 

and define a closed cone by 

(3.3.2) C := {C e RV Q{0 = 0}. 

It follows from Lemma 2.8 that the support of the distribution .F^* -2 F is contained 

2 

in the cone C, for any F e V'^''^. Surprisingly, .F^* -2 F becomes square integrable 

2 
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on C (see Theorem 4.9). As a preparation for the proof, we study a natural action 
on LP'{C) of a parabohc subgroup P"^^^ in this subsection. 
We take a differential (n — l)-form djji on C such that 

dQ A dfj, = d(i A ■ ■ ■ A d(n- 

Then the restriction d/j, to the cone C defines a canonical measure (we use the same 
notation d/i). Using polar coordinates on C: ( = {slu^soj') with s > 0, a; e /S^"^, 
uj' e 5^"^, we write down the canonical measure dfi on C explicitly by 

(3.3.3) j (j)d^=]- [ [ [ (f){su}, sa;')s""^ ds du du' 

for a test function 0. If n > 2, that is if p + g > 4, then the measure d/j, defines a 
Schwartz distribution on M", which is equal to the Delta function S{Q) supported 
on the cone C (see [7]). 

For a measurable function (f) on C, we define a norm of (f) by 

(3.3.4) Uf := I l^l^d/. 

JC 

and denote by L'^{C) = L'^{C\ dfi) the Hilbert space of square integrable functions. 

By the Mackey theory, we can define a natural representation tt of the maximal 
parabolic subgroup = ^maxj^^max^max on L2(C) if p + g e 2Z, by 



(3.3.5)(a) (7r(e*^)V')(C) 
(3.3.5)(b) (7r(m)V')(C) 

(7r(mo)V)(C) 
(3.3.5) (c) (7r(na)V')(C) 



= e-^V(e-*C), (^eR) 

= ^CmQ (m e M--), 

= (-i)'^V'(C) 



Proposition 3.3. 1) The representation (tt, L^(C)) of P™^^ is unitary. 

2) The representation tt is still irreducible when restricted to the motion group 

j^^max^max ~0(p-i,g-i)ix particular, it is irreducible as a P^- 

module. 

Proof. (1) is straightforward from the definitions (3.3.4) and (3.3.5). 
Let us prove (2). It follows from (3.3.5) (c) that any A'"™*^-invariant closed subspace 
of L'^{C) is of the form L^(C") where C is a measurable subset of C. As M^^^ 
acts transitively on C, LP'{C') is M™^^-invariant only if the measiire of C is either 
null or conuU. Thus, L'^{C') equals either {0} or L'^iC). Therefore, the unitary 
representation LF'{C) is irreducible as an M^^^N'^^^-m.odvle. □ 



3.4. It is not clear a priori if (tt, L (C)) extends from pm^x q shall prove 
in Theorem 4.9 that if g > 2 and n{= p + q — 2) > 2 then tt extends to G as an 
irreducible unitary representation through an injective map T: L'^(C) ^ 5'(M"^), 
defined as follows: 

By using the Cauchy-Schwarz inequality, we see the following map 

T(V') : S{W) ^ C, ip^ [ ifipdfx 

Jc 
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is well-defined and continuous if n > 2, for each ip e ^^{C). Thus we have a natural 
map 

(3.4.1) T: L^{C) — > S'(W), il^dfi. 

Clearly, T is injective. We shall regard T{L'^{C)) as a Hilbert space such that T is 
a unitary operator. 

Lemma 3.4. 1) T is a P'^^^ -intertwining operator from {7r,L^{C)) to 
5'(M'^)). 

2) gi pmax , T{L'^{C))) IS ttu irrcduciMe unitary representation of P™^^^. It is 

still irreducible as an M^^^N"^^^ -module. 

Proof. (1) follows directly from the definitions (3.2.1) and (3.3.5). (2) follows from 
(1) and Proposition 3.3 (2). □ 

§4. Green function and inner product 

4.1. In this section, we shall construct solutions of the ultrahyperbolic equation 
□kp-i,9-i/ = by the integral transform given by convolution with the Green 
kernel. Then, we shall show that the Green kernel coincides with a special value 
of the Knapp-Stein intertwining operator for a degenerate principal series. This 
observations gives another expression of the inner product of the minimal repre- 
sentation of 0{p, q) by using the Green kernel (see Theorem 4.7), and also leads to 
a realization of the minimal representation on L'^{C), the Hilbert space of square 
integrable functions on a cone C as will be discussed in §6. 

We put 

p/\ 2, I 2_ 2_ _ 2 

Jr^y^X) — Xi -p * * * -p Xp — 1 Xp ' ' ' Xf^ 

for a; e = W+i-'^. 

4.2. A distribution E satisfying n^p-i,q-iE — S (Dirac's delta function) is called 
a fundamental solution of the ultra-hyperbolic Laplace operator Okp-i.^-i. Recall 
from [7] , page 354, if n is even and n > 2 then 

-r(t-i)e^^=^ 



E = -/I {P{x) + V-10)^-t 

47r 2 

is a fundamental solution of DiRp-i.q-i, where {P{x) -\- \/^0)^ stands for the limit 
of the distribution {P{x) -\- \/— li?(x))''^ as a positive definite quadratic form R{x) 
tends to 0. In view of the integral formula in [7], Chapter III, §2.6 

. n — v'-l'^(g-l) 

47r2e 2 ^ — 



HP + y=To)^-^ = jT^. _ (Q - v^o) 

we have readily the following formula for the Green function Eq of Drp-i,.;- 
Proposition 4.2. We define a distribution Eq on by 

(4.2.1) Eo= ^-={-E + E). 
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Then Drp-i.^-iE^o = and its Fourier transform is given by 



(4.2.2) J^Eo = :r^{{Q - ^0)-' - {Q + ^0)-') = S{Q). 

In the Minkowski case, i.e. q = 2, such a formula has been known ([1]), since 
the so-called two-point functions in the quantum field theory for a zero mass field 
exactly corresponds to {Q + \/—10)~^ for negative frequency. In this case C natu- 
rally splits in two components, a forward and a backward light cone, and functions 
supported on the forward cone have Fourier transforms that extend to holomorphic 
functions on the corresponding tube domain, thus yielding a unitary highest weight 
representation of the connected group. The reproducing kernel of this representa- 
tion is the Fourier transform of the measure on the forward cone, in analogy with 
what happens in Proposition 4.2. 

4.3. In order to give the integral expression of solutions DRp-i.q-i/ = 0, we define 
a convolution map by the Green kernel: 

(4.3.1) S:C^{W)^C°°{W), cp^Eo*(p. 

Fix z/ G C. We consider the representation of G on a subspace 

of C°°(M") (see §2). The restriction to stabilizes C^{W^), as follows from 

(2.7.2) . 

Lemma 4.3. 1) Image 5 C Ker Dikp-i,?-! . 

2) S is an intertwining operator of P^^^ -modules between on one side CTi/+2,e| pmax 
and on the other side tUi^,e| pmax for any u & C and e = ±1. 

Proof. (1) As □jKp-i.g-iE'o = 0, we have Imaged C Ker Drp-i.^-i. 

The proof of (2) is direct from (2.7.2). We illustrate it by the action of e*^ e 

S{w,+2A^'^)<fi)iy) = I Eo{y - ^)e(-+^)V(e*^) dz 

= f Eo{e'y - e*^)e("+2)^+("-2)V(e*2) dz 
= e-*(5^)(e*y) 

This shows that 5" intertwines the action of A"^^^. The case for the action of 
j^maxjymax jg similar and easier. □ 

We shall see that S extends to a G-intertwining operator for = ^ — 1 in 
Proposition 4.6. 

4.4. Recall the notation in §2.9. Let 

[u, v] := UqVq -\ h Up-iVp-i - UpVp Un+lVn+l 

for u,v & = The Knapp-Stein intertwining operator 
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is given by the integral operator with kernel function 



(4.4.1) VA-f ,e(wo - Un+l) = ^A-f ,e([w, Co]) 

on S. Here, we put := *(1) 0, . . . , 0, 1) e S and V'l^^e is a distribution (or a 
hyperfunction) of one variable is defined by 

i/j^Av) '■= p^ 2^l3-. -) l^rXe(sgny). 

Via the bijection ri in the commutative diagram (2.9.4), 

is written as 

{A^J){u) ^ {Ax,J){u) := [ ^x-^,,{[u,v]) f{v) dv {u e M) 

Jm 

in the compact picture M ~ S^~^ x 5"^"^. 

The Gamma factor in the definition of ipy^^{y) exactly cancels the poles of the 
distribution |j/|''Xe(sgn|/) of one variable y with meromorphic parameter v. This 
means that the distribution iIj,^A[u, v]) of multi- variables makes sense for any e C 
when restricted to the open set {{u, v) & M x M : u ±'i'}, where [w, dv] + [v, du] ^ 
0. Then ijj,^^^{[u,v]) continues meromorphically as a distribution on M x M with 
possible poles only atA = 0,— 1,..., whose residues are distributions supported on 
{{u,v) e M X M : u = ±v}. 

In view of the normalization of our parameter (see Lemma 2.9), ^A,e is a G- 
intcrtwining operator from tux+h to zu-x+ii for A 7^ 0,-1,-2,.... What we 
need is the case A = 1 and we recall from [11], §3.9 (basically since we know the 
composition series of the induced representations and the eigenvalue of Ax,e on each 
K-type): 

Lemma 4.4. Let p = q mod 2, g > 2 and {p, q) ^ (2, 2). We put 

(4.4.2) (a) 6:=(-l)"^ 

(4.4.2) (b) 5:=(-l)[^ 

1) The image (n7^_i, Ai^e(C°°(M)e)) is a dense subrepresentation 0/ (tu^'^, V^'^), 

where we recall V^''^ = Ker Am- 

Let us define a Hermitian form ( , )a on the same image AiAC°°{M)^) by 

(4.4.3) (^1,^2).! := S{Ai^^cpi,^) = 6{(pi,A^^), 

for Fi = Ai^^Lpi, G C°°(M)e (i = 1,2), where { , ) is the bilinear form given as 
the integral over M (see (2.10.1)). 

2) The Hermitian form ( , )a is well-defined (namely, independent of the choice 
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ri ^-5 = 0,4 mod 8, 

\ — 1 7? — 5 = 2,6 mods, 

ri p-g = 0,2 mod 8, 

1-1 p - g = 4, 6 mod 8. 



of (pi) and G-invariant under the action of W!±-i. 

3) In comparison with the inner product ( , )m given in Lemma 2.6, we have 
(4.4.4) {FuF2)a = ci(Fi,/2)m for any F^, G ^i,e(C°°(M),), 

where we put 

'^1 • n + l • 

2 2 TT 2 

In particular, the Hermitian form ( , )a is positive definite and the completion of 
a pre-Hilbert space {Ai^^{C°°{M)^), ( , )a) coincides with the Hilbert space Vp^i 
given in §2.6. 

4.5. In the flat picture M^"-'^'^"-'^, we have 

-^0 + ^n+l = Wf-\z"f = P{z) 

for ^ = l{z) by (2.3.4). Then, via the injection r2 in the commutative diagram 
(2.9.4), the Knapp-Stein intertwining operator is given by the convolution: 

(4.5.1) A«> = Ax,eV^ := 2^x-^APi^)) * V 
when restricted to C^(R") C *^(C°°(M)e). Then we have 

2 

(4.5.2) Af^ o *U2 = o Af^. 

'2 2 ' 

Lemma 4.5. Retain the setting of Lemma 4-4- put 



(4.5.3) h{y) := e"-^^(y + v^O)^-t. 
Then we have 

(4.5.4) V'i-t,e(y) = S^^-^ihiy) - h{y)). 



Proof. Solving {y ± ^T^QiY ^ y+ + e^'^'^'y'^, we have 



e 



-i^'^ -e){y + V^Oy - (ev^^'^ - e){y - v^O)^ 



-2v^sin7rzyr(^^^^) 
Then, Lemma 4.5 follows from a residue calculation: 



-l.n _ ^ ^ e-v^aTT _ ^ -(^e 2^"'^ r(^^^) 



lim -n — — - = lim 



i^-T-t siuTTzv r( ^^+j^-^ ) siuTra r(^^^ + f ) TT 



4.6. Our key observation is that the special value of the Knapp-Stein operator 

is given by the Green function up to a scalar constant. In particular, the image 
of C^(M'^) under the integral transform S satisfies not only the ultra-hyperbolic 
equation n^p-i.^-i/ = but also a certain decay condition at infinity that matches 
a conformal compactification of M^"-*^'^"-*^ (see §2.8). 
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Proposition 4.6. Retain the notation as in Proposition 4-2 and Lemma 4-4- 
recall that 5, e = ±1 are determined by p — q mod 8 as in (4-4-2) o-'^d define the 
constant C2 by 

_ 457rtr(^i^) 

r(f-i) • 

Then we have: 

(4.6.1) 7/;i_»,,(P(^)) = C2So, 

(4.6.2) Af^lc-(Rn) =2c25. 

In particular, SiC^iR"")) C ^'^^(yP'^). 

2 

Proof. The first formula follows from Lemma 4.5 and from the definition of Eq 
(see (4.2.1)). The second formula then follows from the definition (4.5.1). Then, 
in view of (4.5.2) and (4.6.2), S extends to a G-intertwining operator between 
(cc7rt+2,^'*^(C~(M)e)) and (run^, ^'*^(C°°(M)e)). Then we have 

2 2 2 2 

S{C^{W)) C ^r,(*U2(C~(M),)) = **^(^f,(C~(M),)) c **^(FP'«). 

'2 2 ' 2 

because C^(R") C **^(C°°(M)e). □ 

2 

4.7. We are ready to introduce an inner product on Ker □jjp-i.g-i by using the 
Green function Eq (see (4.2.1)), so that our minimal representation is realized here 
as a unitary representation. 

We define a Hermitian form on S'(Co°(M"^)) C KerDRp-i.g-i by 

(4.7.1) (/i,/2)Ar:=/ / Eo{y - x)(pi{x)(f2{y) dx dy 

for fi = Sifi = Eq* ifi {1 < i < 2). The right-hand side of (4.7.1) does not depend 
on the choice of (pi because of the formula 

(/l5/2)jV = (/l, <^2)l2(M") = (<^l5 /2)L2(]R")• 
We recall from Proposition 4.6 and Lemma 2.8 the inclusive relations: 
(4.7.2) 

5(Co~(M^)) C ^f,(**^(C~(M),)) C **^(F^'^) C KerDR.-i,,-! C C~(R"). 

' 2 2 

Theorem 4.7. Let p = q mod 2, p,q>2 and {p, q) ^ (2, 2). Recall n = p + q — 2 

and e = (— 1)^2"^ . 

1) The Hermitian form ( , )n is positive definite on S{C^{W^)). 

2) The Hermitian form ( , )n is invariant under ujn^ ^ ^^(G). 

Let 7i be the completion of the pre-Hilbert space {S{Cq° {MJ^)), ( , )n)- 

3) The Frechet representation of G on ^* -2 {C°°{M)f) induces a repre- 

2 ' ' 2 

sentation of G on the Hilbert space H, which we shall denote by {w^p^i^g_i,Tt) . 
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4) The unitary representation (■Cc7^p"i_q_i, 7i) is irreducible. 

5) The map (see (2.8.7) for the definition) 

2 

n n 

C'^{W) C°°{M) 

extends uniquely to a unitary isomorphism between (ti7^p°i,<,_i, 7i) and the minimal 
representation {vo^'^, yp.^) up to a scalar constant. More precisely, for any /i, /2 £ 
Ti, we have 

(4.7.3) (/i,/2)Ar = 22-(Fi,F2)M, 

where we put 

:= (vP*^ J-V,, (i = l,2). 

2 

Remark. 1) We shall give a diflFerent proof of the first statement of Theorem 4.7 in 
Theorem 4.9 by using the Fourier transform of the Green kernel. 

2) As we shall see in the proof of Theorem 4.7, S{C^{W^)) is dense in (V^^'") 

2 

with respect to the above inner product. 

3) We can realize naturally the Hilbert space 7i as a subspace of the Schwartz 
distributions 5'(R"^), namely, JF~^T(L^(C)), as we shall prove in Theorem 4.9. 

4.8. In order to prove Theorem 4.7, we need: 

Lemma 4.8. Retain the notation of Theorem J^..!. 

1) For any fi,f2 e S{C^{W)), we have 

(4.8.1) (/i,/2)iv = C3(Fi,F2)^, 

where 

_ 7rt£(^iJ±^ 

C3 :- r(f-i) • 

We extend the Hermitian form {,)n from S{C^{W)) to (C°°(M)e)) 

' 2 

by using the right side of (4-8.1) (recall the inclusive relation (4-7.2)). 

2) The Hermitian form ( , )n is positive definite on ^^^(^* +2 iC°°iM)f)), in 

' 2 

which S{Cq^{W^)) is dense. In particular, the Hilbert space Ti (see Theorem 4-7) 

coincides with the completion of the pre-Hilbert space ^i,e(^* +2 iC°°{M)^)). 

' 2 

We first finish the proof of Theorem 4.7, and then give a proof of Lemma 4.8. 

Proof of Theorem 4-'^- (1) is clear from Lemma 4.8 (1) and Lemma 4.4. Next, let 
us prove (4.7.3). We consider the G-intertwining operator 

(**_2 )-' : (ylf,(C~(M),)) ^ Af,(C~(M),) C V^-'?, 

2 ,e 2 ' 

or equivalently by (4.5.2), 

(**_2 J-^ : f*U2(C~(M)e)) ^ F^'^. 
2 ' V 2 / 

23 



Combining (4.4.4) and (4.8.1), we have 

(/i,/2)iv = C3(Fi,F2)a = ciC3(Fi,F2)m = 2''-'' {F,, F^) M , 
where the second equahty follows from a classical formula of the Gamma function: 

Thus, we have proved (4.7.3), especially, (^^^^-2 is an isometry up to scalar. 
Then (^^^-2 )~^ extends to an isometric (up to scalar) G-intertwining operator 

2 

H — > VP'i. This map is surjective because {wP''^,VP'1) is an irreducible unitary 
representation of G (see Lemma 2.6). All other statements are now clear. □ 

Proof of Lemma 4.8. 1) Suppose fi = S(pi, (pi e C^{W) {i = 1,2). Recall the 
constant C2 in Proposition 4.6, we have 



(/i,/2)iv==/ {SLpi){x)(p2{x)dx 

f «>i)(a;)^^cZx 
(4.8.2) = -L /■ (Af,((*U2 ,)-Vi)(?') (*k. J-'Mb) db. 

4C2 Jm 2 2 

We put 

V'i:=-(*U2 J-Vi eC~(M)„ (i = l,2). 

C2 2 

Then Fi = A^^i/ji by (4.5.2). Therefore, 

(4.8.2) = j^iKMih) M^db^ ^(Fi, F2)^ 

which equals the right side of (4.8.1). 

2) It is enough to show that »S'(Co^(R"^)) is dense in a pre-Hilbert space 

^i,e(^ti+2 (C'°°(M)e)) because the inner product ( , )a is positive definite from 

' 2 
Lemma 4.4. 

Suppose / = Ai^^(p {(p e 5'*^(C°°(M),)) is orthogonal to 5(C^(R'^)) with 
' 2 

respect to the inner product ( , )a- This means that {f,Ai^^(f))A — for any 
e C^(M^). Then we have 

/ (**:^ ,)"'/(&) (**.-2 db = forany(/.eCo°°(M"). 

Since (**^ J-i(C^(R^)) is dense in L^{M)^ := {/ e L^{M) : /(-«) = €/(«)}, 

we have (*t^ )-V = 0, and thus / = 0. □ 

2 

4.9. We recall *t^(Ff'«) C *t^(C°°(M)e) C 5'(R^), on which we can define 

2 2 

the Fourier transform J^. We consider the following maps (see §4.3, §3.2, §3.4): 

2 

Here is a description of the Fourier transform of the minimal unitary representation 
(tu^p"i , 7i) of G = 0{p, q) which is obtained as the completion of a pre-Hilbert 
space (S'(C^(M")), ( , )n) (see Theorem 4.7). 
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Theorem 4.9. 1) J^5(C^(M")) is contained in T{L^{C)). Furthermore, it is 

dense in the Hilhert space T[L'^{C)). 

2) T-^ oT: 5(C^(M")) ^ L^{C) extends uniquely to a unitary map H L^{C) 
up to a scalar constant. This constant is given explicitly by 

(4.9.1) (27r)"(/,/)^ = ||T-io^/||i.(^). 

3) The P"^'"^^ -module (tt, L^(C)) extends to an irreducible unitary representation of 
G, denoted by the same letter , so that {2n)~^T~^ o gives a unitary equivalence 
between {w^p^i^g^ijTi.) and (tt, L^(C)). 

4) T~-^ o^o^f^j_2 : V^'"^ -^^(C*) induces a unitary equivalence between irreducible 

2 

unitary representations {wP''^,VP'1) and (tt, L^(C)), up to a scalar constant given 
by: 

(4.9.2) (0, 0)m = 47r"||T-i o ^ o ^l-^nh^c) 

Proof. U f = Eo*(p {(fi e Cq° (]£"■)), then it follows from the integratfon formula of 
the Green function (see Proposition 4.2) that its Fourier transform is given by 

Tf = T{Eo * <fi) = {TEo){T<fi) = {T<fi)S{Q) = T{{T<p)\c). 

Since ip E C^(M'^), we have {J^^)\c e ^^(C). Hence, J^S{C^{W)) is contained in 
T(L2(C)). Then we have (4.9.1), as follows from 

(4.9.3) WiJ^Mll^c) = {{rEo){T^),r^) = {J'{Eo*<p),J'<p) = (27r)-(/, /)^r. 

We note that (4.9.3) gives a different proof that ( , ) at is a positive definite Her- 
mitian form on S{Cq° (W^)) (see Remark after Theorem 4.7). 

It follows from Lemma 3.4 that T~^ o JF is an P™^^-intertwining operator from 
(wn-2 g I pmax , 5* (Cg" (R" ) ) ) to {tv , L"^ (C)) . THs map is isometric up to a scalar 

by (4.9.1). Then, it extends naturally to an P™*^-intertwining operator from 
{w^p^i^q-i I pmax , H) to (7r,L^(C)), which is surjective because (7r,L^(C)) is irre- 
ducible (see Proposition 3.3). Hence we have proved (1), (2) and (3). The statement 
(4) follows from (2) and Theorem 4.7. Thus, we have proved Theorem 4.9. □ 

4.10. By semisimple theory, it is known that a minimal representation is still 
irreducible when restricted to any maximal parabolic subgroup. In particular, tu^'^ 
is irreducible as an P™*^-module. In our case, this fact can be strengthened as 
follows: 

Corollary 4.10. The restriction of the minimal representation zo^''^ (equivalently, 
ti7™'^i,,_i) to M^^^'W^ ~ 0{p -l,q-l) X RP+«-2 is still irreducible. 

Proof. Theorem 4.7 and Theorem 4.9 show that w^''^, ti7^p"i,q_i and tt are unitary 
equivalent to one another. Now, Corollary follows from Proposition 3.3. □ 
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4.11. For the convenience of the reader, we summarize the maps used in the 
proofs. 



n-2 



(4.11.1) C°°(M) ^ C°°(R")n5'(R") -^S'(W) 
U U U 

2 2 

n dense fl dense fl dense 



(w^.^VP.'z) ^ ^T(L^(C)) ^(7r,L^(C)). 

In the last hne, we have written also the notation for unitary representations. 
§5. Bessel function and an integral formula of spherical functions 

5.1. In this section, we shall compute explicitly the lowest JC-type of our minimal 
representation in the A'"-picture, i.e. find it as a solution to D^p-i,q-if = 0, and also 
its Fourier transform as a function on C; this turns out to be written in terms of a 
Bessel function (see Theorem 5.5). Note that except when p — q we are not dealing 
with a spherical representation of G (namely, there is no non-zero i^-fixed vector 
in our representation). At the end of this section, we reformulate the equivalent 
realizations of the minimal representation found in the previous section, using now 
the minimal K-type to understand the different pictures. 

5.2. Without loss of generality, we may and do assume p > q in this section. 
Instead of K-fixed vectors, our idea here is to focus on an 0{p) x 0{q — l)-fixed 
vector. Then, it follows from Lemma 2.6 that such a vector, which we shall denote 
by Fq, is unique in our minimal representation (^17^'^, V^''^) up to a scalar multiple, 
and is contained in the minimal K-type of the form 1 Kl ?Y^^(M^). (We note that 
this -ftT-type is not one dimensional if p ^ q). 

We shall find an explicit formula of the Fourier transform of this vector Fq after 
a conformal change of coordinates. We start with the following classical lemma, 
for which we give a proof for the sake of the completeness. We take a coordinate 
(fXp+i, . . . , Wp+g) in IR^ and realize 0{q — 1) in 0{q) such that it stabilizes the last 
coordinate Up+q. 

Lemma 5.2. For any I e N, 0{q — l)-invariant spherical harmonics of degree I 
form a one dimensional vector space. More precisely, we have 

~ r w g-2 + Z g-1 ^ ^ 
K[K^) ~(L2f'i(^, 2 '^P+i -I HWp+q-i j. 

Proof. In terms of the polar coordinate of 5'^"^: 

(5.2.1) $++ : 5^-2 X (0, |) ^ S'^-\ {y,e) ^ ((sin^)?/, cos^), 

the Laplace-Beltrami operator on S^~^ takes the form: 

(8.2.2) A,,.. = ^ + - 2) cot e| + ^ A,..,. 
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If F e T-C''{W) is 0(^ — 1) -invariant, then Fo$_|__|_(6', y) depends only on 9, for which 
we write h{9). Then h{9) is an even function satisfying: 



+ (9 - 2) cot 9-^ + l{l + q- 2))h{9) = 0. 
Since h{9) is regular at 6* = 0, it is a scalar multiple of the Jacobi function: 

/ , a — 1 1 — cos 9 , 
= 2FiH,Q-2 + /;^; ). 

Thus, we have proved the lemma. □ 

5.3. In view of Lemma 2.6, the special case of Lemma 5.2 with / = yields: 

Proposition 5.3. Suppose p>q>2,p + qE2N and {p,q) ^ (2,2). Let 
(•ui, . . . , Upj^q) be the coordinate of M — Sp~^ x S'^~^ in IR.^+^. We define a function 
Fo-.M^Cby 

(5.3.1) Fo(wi,...,Wp+g) :=2i^i(^^, ^ ;Wp+i H hWp+g-i). 

Then Fq is an 0{p) x 0{q — 1) -invariant analytic function on M satisfying the 
Yamabe equation AmFq = 0. Conversely, any such function is a scalar multiple of 
Fo. 

Remark. U p + q is odd, then Fq in the right side of (5.3.1) still gives a solution 
^mF = on an open dense set of M such that ttp+q 7^ 0. Furthermore, Fq is a 
continuous function on M. However, it does not solve the Yamabe equation as a 
distribution on M. 

5.4. We recall t{z\z") is a conformal factor defined in (2.8.1). Let us define 

/o := ^t^Fo, namely, 
2 

(5.4.1) f^{z',z"):=T{z',z") - 2Fi(^, ^ '^^'^ r{z'!z"f ^- 

We note that \t{z' ,z")\ > \z"\ for any {z',z") e MP-^''?-^ The equality holds if 
and only if l^'p — \z"\'^ = —4. 

The following Proposition is immediate from Lemma 2.8 and Proposition 5.3: 

Proposition 5.4. With the same assumption onp,q in Proposition 5.3, we have: 

1) fo is a real analytic function on that solves Dkp-i, 9-1/0 = 0. 

2) fo is 0{p) X 0{q — l)-invariant. 

We say Fq is the generating function of V^''^ — Ker Am, and fo is that of 

Remark. More strongly than Proposition 5.4, one can prove that fo is a real analytic 
solution of DKp-i.q-i/o = 0if7?-|-q'>4by using Proposition 5.6, where we do not 
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assume that p + q is even. The real analyticity is not obvious from the expression 
(5.4.1) in the neighbourhood of the hypersurface of l^'p — Iz"]"^ = —4. 

5.5. We recaU the definitions of Bessel functions: 

m=0 ' 



m=0 



-^i^f-^) = / — r-^S r (modified Bessel function of the first kind), 

^ ^ ^ m! r(z/ + m+ 1) ^ ^' 



2 si 
Then satisfies 



Ki,{z) = — — - — : — ^ (modified Bessel function of the second kind). 



The asymptotic of the functions K^, is well-known, for example z~'''K,y{z) decays 
exponentially as z ^ +oo. 

Theorem 5.5. We put \C\ := (Ci^ + ■ ■ ■ + Cn^)^ for C= (Ci, ■ ■ ■ Xn) e W . Let 

Fq e C°°(M) be the generating function of V^''^ — KerAjw" (see Proposition 5.3). 
Then 



p+g — 2 p—5 r(^-2 ) ^ , ^ 3 — 9 

" r(^ 2 



(^*t^Fo)(C) = (27r)^2-^ ^^;^^4J C|^K^(2|C|)(^(g) 



Note that the \C\^ K ^-3 (2\C\) belongs to the Hilbert space L'^{C) if p + q > 4 
by the asymptotic behaviour of the Bessel function Ki, and by the explicit form of 
5{Q) in (3.3.3). 

5.6. Theorem 5.5 follows from the following Proposition: 

Proposition 5.6. We write for the inverse Fourier transform. With notation 
in (5.4.1), we have 

(5.6.1) J^-' (\C\'^K^{2\CmQ)) (z) = -iT3^1^^7-T-/o(^). 

^ ^ 22772! (^^) 

Proof. Let (/)(r) be a function of one variable, which will be taken later to be 
r^K^{2r). Then, it follows from (3.3.3) that for z = {z',z") e RP-'^'I-'^, 

= (27r)-(f+«-2) Iff (/,(r)e-^«^''^'^)+(^"'^^)) r^+'^'^dr dw drj 
Using the formula jgm-i e^^^^'^-'^^cia; = (27r)^t^~^ Jm_i(t), we have 

l-OO 

= (277)-"^ / (i){r){r\z'\)^-^Jp^{r\z'\){r\z''\)^-^J^{r\z''\)rP+'i-^dr 

/>oo 

={2tt)-'^\z'\^\z"\^ / (i){r)Jv^{r\z'\)J,_^{r\z"\)i 

Jo ^ ^ 



P + q-4 

)r 2 dr. 
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Now, put 0(r) := ■2'' K q^{2r). We use the following formula of the Hankel 
transform due to Bailey [2] (see also [6], §19.6 (8)) 



t^~^J^{at)Ji,{bt)Kp{ct)dt 
2A-2^M;,^r(|(A + fM + i^- p))r(i(A + + + p)) 



cA+M+^^r(/i + i)r(zy + 1) 



Here, F4 is the Appell hypergeometric function of two variables, defined by 



Then we have: 



r-OO 

^{27t)-^\z'\^\z"\^ / K^{2r)J^{r\z'\)J^{r\z"\)r'^ dr 

Jq 2 2 2 

.o,^-^o^ ^(^) p-1 p + q-i p-l q-l_-\z'\\-\zr. 
= (2^) 2 ^^^^ ^ 



5.7. Then, the proof of Proposition 5.6 will be finished by showing the following 
reduction formula: 

Lemma 5.7. Let t{z', z") he the conformal factor defined in (2.8.1). We have 

p-1 p + q-4: p-1 q-1 -\z'\^ -W\^ . 
2 ' 2 ' 2 ' 2 ' 4 ' 4 ^ 

(5.7.1) =r(.,.) ^ ^^i(^'^^;^;^(;ri^)- 

Proof of Lemma 5.7. We recall a reduction formula of Appell's hypergeometric 
functions (see [5], §5.10, (8)): 

(5.7.2) F4{a,(3-l + a- (3,p- ~^ ~^ ^ 



{l-x){l-yy {l-x){l-yy 
= (1 - 2/)"2Fi(a,/?; 1 + a - /3; Z^ilz^) 

and a quadratic transformation for hypergeometric functions (see [5], §2.11 (32)): 
(5.7.3) 2Fi(«, 1 + a - /5; ^) = (1 - ^)-^Fi(-, ^ ^ ' 1 + « - Z^; (13^)2 )• 
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Combining (5.7.2) with (5.7.3) for a = ^"'"^ ^ and /3 = and using the symmetry 
of a and 6; (c, x) and (d, y) in ^4(0, 6; c, cZ; x, y), we have 

p — 1 p + q — 4 p — 1 q — 1 —X —y ^ 



2 ' 2 ' 2 ' 2 '(l-x)(l-2/)'(l-x)(l-y)^ 

^ p ( P + q-4: q-p q - 1 4y{l - x){l - y) 
^ 4 ' 4 ' 2 ' (l-xy)2 ^ 

If we put 



4 (l-a;)(l-y)' 4 (l-a;)(l-y) 
then a simple computation shows 

// //n2 f ^-xy V k'T 4y(l -a;)(l -y) 

t{z ,z ) = - 



{l-x){l-y)J ' r{z',z")^ (l-xy)^ 
Thus, Lemma 5.7 is proved. □ 

5.8. Using the generating function Fq we may recover the whole representation 
by letting the Lie algebra of G act. Let us see how our previous results may be 
reformulated: It follows from the definition of ^ (see §3.2) that the linear map 

J^o^l^ : C°°(M) ^ 5'(R") 

2 

induces a natural intertwining map from (ru^'^, F^'^) to {wn^ ^, .F^ti-2 jV^''^)) as 

2 ' 2 

G-modules and also as g-modules. Here again e = (— 1)^~, F^'^ = Ker Am (c 

C°°(M)) and ^l^iVP'i) C KerDKP-i.^-i. 
2 

For b e R"^, m e 0(p — l,q — 1), we put a function on the cone C by 

(5.8.1) iPb,m{C) :=e^<'''^>|mCl^i^s^(2|mC|). 

In particular, we have 

V'0,e(C) = ICl'^i^^(2|CI)- 

Here we give explicit functions which are dense in the minimal representations. 

Theorem 5.8. Suppose p> q>2, p + q e 2N, and (p, q) ^ (2, 2). 

1) V'o.elC) ^ K-finite vector of {tt,LP'{C)). It belongs to the minimal K-type of 

TT. 

2) C-span{^|Jb,m '■ b G M", m e 0(p — 1, g — 1)} is a dense subspace of the minimal 
representation {tv,L^{C)). 

2') C-span{i(^i,^rnS{Q) : b e W^,m e 0(p — l,q — 1)} is a dense subspace of the min- 
imal representation (wri^ J^^'*-2 (T^^'^)). 

2 ' — 2 — 

Proof. This follows by combining Theorem 4.9 and Lemma 5.5, and using that by 
Mackey theory we have an irreducible representation of the parabolic group. □ 
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5.9. The advantage in the reahzation on L'^{C) is that the action for pmax ^^^^ ^j^g 
inner product are easily described. On the other hand, the action of K is not easy 
to be described, and especially, the K-finiteness in the statement (2) is non-trivial. 

Let U (0) the enveloping algebra of the complexified Lie algebra Q. We define a 
subspace of 5'(M") by 

(5.9.1) U := dwr^^^{U{Q)){i;o,eSiQ)). 

That is, t/ is the linear span of a Bessel function \C\^ K q^{2\(\)dii on the cone 
C and its iterative differentials corresponding to the action of the Lie algebra g. 

We have seen that C T{L'^{C)) in Theorem 4.9. We may restate 

2 

in this way: 

Theorem 5.9. 1) U is an infinitesimally unitary {q, K) -module via UJ n — 2 . 

2) U is dense in the Hilbert space T{L^{C)). 

3) The completion of (1) defines an irreducible unitary representation of G on 
T{L'^{C)), and then also on L'^iC). This gives an extension of tt from P™ax q 

This has already been done by Theorem 4.9 and the irreducibility of the minimal 
representation. 

One of non-trivial parts of the above assertion is to show 

Uf\T{L^{C)) {0} 

which was proved in Theorem 5.5. 

§6. Explicit inner product on solutions r\-^v-^,<i-^f = 

6.1. The aim of this section is to provide an explicit inner product on a certain 
subspace (see (6.2.1)) of solutions of the ultrahyperbolic equation D^p-i.^-i/ = 0, 
such that its Hilbert completion gives the unitarization of the minimal representa- 
tion of 0{p, q). 

Roughly speaking, the inner product will be given in terms of the integration 
over a hyperplane after convoluting a distribution along the normal direction. 

We assume n > 2. We fix i e {1,2, . . .,n} once and for all. The hyperplane 
on which we integrate will be {2; = (2:1, ... , Zn) G R" : Zi = 0}, for which we simply 
write W-'^. 

6.2. Let C*^*) := {C e C* : Ci 7^ 0}? open dense subset of the null cone C (see 

(3.3.2) ). We note that C^{C \ {0}) is dense in L^{C). We define a subspace of 
solutions of Ker Dkp- 1,9-1 by 

(6.2.1) W :=T-^oT{C^{C^'^)). 

Here, we recall T : L^{C) ^ 5'(M'") is the embedding via the measure d/U on the 
cone C. By the Paley- Wiener theorem for compactly supported distributions, W 
consists of real analytic solutions of Dkp- 1,9-1/ = 0. 

Using an interpretation of the Dirac delta function in terms of hyperfunctions: 

~ 27rv^ (^i - ~ Zi + x/^o) ' 
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we decompose f eW as 



(6.2.2) f{z)^f^\z)+f^_!\z), 

where f^^ (z) is defined by the convolution in the 2;j-variable: 

(6-2.3) f^\z) := • * f{z). 

We shall see later that the decomposition (6.2.2) makes sense not only for / e 
but also for any / e o T{L'^{C)) (see Lemma 6.5). We set 



(6.2.4) {fJ)w:=4Tl I , [ff^-f^^^\\z,=odz,---dz,---dzn 



Theorem 6.2. Fix any i G {1, 2, . . . , n}. 

1) The formula (6.2.4) defines a positive definite Hermitian form on W , a subspace 
of solutions of the ultrahyperbolic operator n^p-i,q-i = + ■ — h — -^-^ — 

^ , on RP-i'^-i. 

2) The inner product (6.2.4) is independent of the choice of i & {1,2,..., n}. 

3) The action ofG = 0{p, q) preserves the inner product (6.2.4), that the Hilbert 
completion WofW defines a unitary representation of G = 0(p,q). 

4) The resulting unitary representation is unitarily equivalent to the minimal rep- 
resentation (ci7^'^, yP'5). The G -intertwining operator : V^^'^ —>■ W gives a 

2 

unitary equivalence up to a scalar constant. 

5) cJF~^ o T : LP'{C) W is a unitary G -intertwining operator, if we put c = 

r, n+2 n + 1 
2 2 TT 2 . 

This Theorem gives a new formulation of the Hilbert space of the minimal rep- 
resentation purely in terms of intrinsic objects in the flat space M^"^'^"-*^ where the 
differential equation is the classical ultrahyperbolic one. It generalizes the q = 2 
case where an inner product was known in terms of integration of Cauchy data - 
here one could interpret the inner product in terms of the energy generator. The 
interesting property about the inner product is its large invariance group; even 
translational invariance amounts to a remarkable "conservation law" , and we may 
also note that the integration over a coordinate hyperplane can be replaced by 
integration over any non-characteristic hyperplane (since such a hyperplane is con- 
jugate to either = or 2;^ = by 0{p — 1, q — 1) x ~^''3'~^), or even the image 
of such a hyperplane under conformal inversion. 

The strategy of the proof of Theorem 6.2 is as follows: We recall from Theo- 
rem 4.7 that the Hilbert space H is the completion of the space S{C^(R'^)) with 
respect to another inner product ( , )Ar (S* is an integral transform by the Green 
kernel). Since C^(C\ {0}) is dense in L'^{C) and since T-^oJ^-.H-^ L^{C) is an 
isomorphism of Hilbert spaces (up to a scalar) by Theorem 4.9 (1), is a dense 
subspace of the Hilbert space (7i, ( , )n)- In light of this, the key ingredient of the 
proof of Theorem 6.2 is to give a formula of ( , )w by means of ( , )l'^(c)- We shall 
prove: 

(6.2.5) 2{27rr+\f,f)w = 11^"^ o for any feW. 
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Once we prove (6.2.5), it follows from (4.9.1) that 

47r( , )m/ = ( , )n 
on the subspace W. In particular, we have 

n = w, 

and all other statements of Theorem 6.2 on our inner product { , )w ^re clear from 
the corresponding results on the inner product ( , )n proved in Theorem 4.7 and 
Theorem 4.9 (e.g. Theorem 6.2 (4) corresponds to Theorem 4.7 (5); Theorem 6.2 (5) 
to Theorem 4.9 (3)). 

The rest of this section is devoted to the proof of the formula (6.2.5). 

6.3. We define an open subset of the cone C by 

:= {C e C7 : ±Ci > 0}. 

Then 

is an open dense subset of the cone C, and we have a direct sum decomposition of 
the Hilbert space: 

We define the Heaviside function Y^^ {Q of the variable Q by 

For 0eC^(CW), weput 
Then Supp05J^ C and we have 

(6.3.1) 4> = (t>^;^ +4>''i\ 

(6.3.2) mhic) = Uf\\Uc) + \\4>^-\\hicy 

6.4. Let us take the n — 1 variables Ci , . . . , ^j, . . . , as a coordinate on Cj_ . Then 
we have 

(6.4.1) Ci = ±v'g(^)(Ci,...,5,...,Cn) 

on C^^ , respectively, if we put 

(6.4.2) Q«(Ci, . . . , S, . . . , Cn) := -Si (Ci' + C2' + • • • ±C? ± • • • - Cn-l' - Cn') , 
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where Si = ±1 is the signature of in the quadratic form Q(C) as in (2.5.1). We 
note that Q*-*^ (^i, . . . , ^j, . . . , > on the cone C, and the map 

(Cl,...,0,---,Cn) H-^(Cl,---,±yQ(^,...,Cn) 

gives a bijection from {(Ci, . . . , 0, • • • , Cn) e M'" : Q^^^ (Ci> • • • , • • • , Cn) ^ 0} onto 
C''*^. By substituting (6.4.1) into (f)^£ , we put 

^5!)(Cl,...,G,...,Cn) := </.2^(Cl,...,±A/Q«,...,Cn). 

Since the measure d/j, on the cone C is of the form ^^J—-— d(i ■ ■ ■ dQ ■ ■ ■ d(n, we have 



(6.4.3) 



I 



dC,i- ■ -dCi- ■ ■ dC„ 



1 



X e 



(27r)' 
tV- 



Cl H \-ZiCi-\ \-ZnCn) 



1.,VQ^ V± (Cl,..-^---,Cn) ... ^. ... 

2yg« 



From (6.4.4) we have the foUowing: 

Lemma 6.4. We write for the Fourier transform in'MJ' (k = n — l,n). Then, 



{zi, . . . , Zi, . . . , Zfi), 



d_ 
dz^ 



T, \zi=0^ ^nT(j)^_J — - JF (^^±^ (^1, • • • , ^i, • • • , ^n)- 



47r 



6.5. We recaU the Fourier transform of the Riesz potential 

/oo 
-oo 

for a meromorphic parameter A. Letting A = 0, we have 

^ ^^i^i T ^Or'S{z^, ...,Zi,...,Zn), 

where ^(2:1, . . . ,Zi, . . . , Zn) is the Dirac delta function of n — 1 variables. Then we 
have 



27r 
±1 



27rv^ 



Zj^V^O) ^5{zi,...,Zi,...,Zn)* f{z) 

{zi^^Q)-Uf{z). 



Here the first two convolutions are for n variables zi, . . . ,Zn, while the last one for 
only Zi e M. In view of the definition of and f^ , we have proved 
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Lemma 6.5. /// = J^'^Tcj) then = T'^Tcj)^^ . 

6.6. We are now ready to complete the proof of Theorem 6.2. By using Lemma 6.4 
and Lemma 6.5, the Plancherel formula for M"^~^, and the integration formula 
(6.4.3), respectively, we have 



Qi"(*) I / T (*) 



Hence, 

= 2(27rV+^ (11-^+ lli^co + 110- lll^cc)) = i^^^y^\m\L^{cy 

This finishes the proof of the formula (6.2.5) and hence Theorem 6.2. □ 

6.7. The main content of Theorem 6.2 is to give yet another realization of the 
inner product and of the Hilbert space. This is very close to the form most known 
in the case of the wave equation, where one integrates Cauchy data on the zero 
time hypersurface to get the inner product. Note the connection to the theory of 
conserved quantities for the wave equation - we end the paper by making more 
explicit this final remark: 

When p = 2 let us introduce time and space coordinates (t, x) by 

(t, x) = (t, Xi, . . . , Xn-l) = {zi,...,Zn) 

and the dual variable A; to x so that positive-energy solutions to the wave equation 
are given by the Fourier transform {i = -\/— T) 

u+{t,x)= [ e'^^--'\^\)'^^dk 
7m"-i I"'I 

and similarly for negative-energy solutions 

Jk"-i 1^1 

where kx denotes the usual scalar product, \k\ the Euclidean length, and dk = 
dki ■ ■ ■ dkn-i- Any solution is the sum of two such: u = u'^ + u~ . The energy of 
the wave u is given by 
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and is a conserved quantity, i.e. independent of which constant-time hyperplane we 
integrate over. It is easy to see that cross-terms drop out, so that on the Fourier 
transform side we obtain 



for the energy. Note that this energy only differs from our inner product by a 
density factor of \k\, and that it may be thought of as an integral over C. In terms 
of our conformally invariant inner product (6.2.4) this is up to a constant just 



where H = idt is the energy generator (infinitesimal time translations). In the 
same way, we have the analogous "conserved quantities" for the ultrahyperbolic 
equation and the inner product (6.2.4), namely: Let Hj = idz^ be the generator of 
translations in the coordinate Zj , then for a solution / in the Hilbert space 



is invariant under translations in the coordinate Zj. Furthermore, the quantity £j (/) 
can be expressed in terms of an integral of local quantities. In particular we may 
use this to study uniqueness and decay properties of solutions to □jKP-i.g-i/ = 0. 
Since this is outside the scope of the present paper, we shall not do so here; but note 
that one easy consequence is the fact, that if a solution and its normal derivative 
vanish on a coordinate hyperplane, then it is identically zero - a classical fact about 
the energy (time zero hyperplane). 
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